Vision-Based Sign Language Recognition Using Sign-Wise Tied Mixture HMM

In this paper, a new sign-wise tied mixture HMM (SWTMHMM) is proposed and applied in vision-based sign language recognition (SLR). In the SWTMHMM, the mixture densities of the same sign model are tied so that the states belonging to the same sign share a common local codebook, which leads to robust model parameters estimation and efficient computation of probability densities. For the sign feature extraction, an effective hierarchical feature description scheme with different scales of features to characterize sign language is presented. Experimental results based on 439 frequently used Chinese sign language (CSL) signs show that the proposed methods can work well for the medium vocabulary SLR in the unconstrained environment.